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In this paper we address the problem of efficient estimation of Sobol sensitivy 
indices. First, we focus on general functional integrals of conditional moments 
of the form E(^(E(99(y)|X))) where {X, Y) is a random vector with joint den- 
sity / and ip and y? are functions that are differentiable enough. In particular, 
we show that asymptotical efficient estimation of this functional boils down 
to the estimation of crossed quadratic functionals. An efficient estimate of 
first-order sensitivity indices is then derived as a special case. We investigate 
its properties on several analytical functions and illustrate its interest on a 
reservoir engineering case. 

Keywords: density estimation, semiparametric Cramer-Rao bound, global 
sensitivity analysis. 
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1. Introduction 

In the past decade, the increasing interest in the design and analysis of co mputer experi- 



ment s motivated the development of dedicated and sharp statistical tools (jSantner et al.. 



20031 ). Design of experiments, sensitivity analysis and proxy models are examples of re- 
search fields where numerous contributions have been proposed. More specifically, global 
Sensitivity Analysis (SA) is a key method for investigating complex computer codes 
which model physical phenomena. It involves a set of techniques used to quantify the 
influence of uncertain input parameters on the variability in numerical model responses. 
Recently, s ensitivity studies have been app lied in a large var iety of fields, ran ging from 
chemist ry (Cukier et al.lll973l : lTuranyilll990 ) or o il recovery ( looss et al.ll201ll ) to space 



science ( Carrasco et al.ll2007l ) and nuclear safety ( looss et al 



20061 '). 



In general, global SA refers to the probabilistic framework, meaning that the uncertain 
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input parameters are modelled as a random vector. By propagation, every computer code 
output is itself a random variable. Global SA techniques then consists in comparing the 
probability distribution of the output with the conditional probability distribution of the 
output when some of the inputs are fixed. This yields in particular useful information 
on the impact of some parameters. Such comparisons can be performed by consider- 
ing various criteria, each one of them providing a different insight on the input-output 
relationship. For example, some criteria are based on dist ances between the probability 
density functions (e.g. L^ and L^ norms ( Borgonovd ( 20071 )) or Kullback-Leibler distance 
( Liu et al.l ( 20061 )). while others rely on functional s of conditional mom ents. Among those, 
variance-based methods are the most widely used ( Saltelli et al.ll2000l ). They evaluate how 



the in 
dices ( 



3uts contrib ute to the output variance through the so-called Sobol sensitivity in- 



SoboT 199j), whi ch naturally emerge from a functional AN OVA decomposition of 



the output ( Antoniadiall984l : iHoeffding 1948 : Owen 1994 ). Interpretation of the indices 
in this setting makes it possible to exhibit which input or interaction of inputs most 
influences the variabil ity of the computer code out put. This can be typi cally relevant 
for m odel calibration ( Kennedv and O'Haganll200ll ) or model validation ( Bavarri et al. 
20071 1. 



Consequently, in order to conduct a sensitivity study, estimation of such sensit ivity in- 



dices is of great i nterest. Initially, Monte-Carlo estimates have been propo sed (IMcKav 



1995 : ISobollll993l ) . Recent work also focused on their asymptotic properties ( Janon et al. 



20121 ). However, in many applications, calls to the computer code are very expensive, 
from several minutes to hours. In addition, the number of inputs can be large, making 
Monte-Carlo approaches untractable in practice. To overcome this problem, recent work 
focused on the use of metamodeling techniques. The complex computer code is approx- 
imated by a mathematical model, referred to as a "metamodel", which should be as 
representative as possible of the computer code, with good prediction capability. Once 
the metamodel is built and validated, it is used in the extensive Monte-Carlo sampling 
instead of the complex numeri cal model. Several metamod el s can be used: pol ynomials, 
Gaussian pro cess metamodel s ( Oaklev and Q'HaganI (J2004l ). llooss et al.l (J201ll )) or local 
polynomials (JDa Veiga et al.l (120091 )). However, in these papers, the approach is gener- 
ally empirical in the sense that no convergence study is performed and do not provide 
any insight about the a symptotic behavior of the sensitivity indices estimates. The only 
exception is the work of lDa Veiga et al.l ( 20091 ). where the au thors investigate t he co nver- 
gence of a local-polynom ial based estimate using the work of lFan and Gijbelsl (jl996l ) and 
Wand and JonesI ( 19941 ). In particular, this plug-in estimate achieves a nonparametric 



convergence rate. 

In this paper, we go one step further and propose the first asymptotically efficient esti- 
mate for sensitivity indices. More precisely, we investigate the problem of efficient esti- 
mation of some general nonlinear functional based on the densit y of a pair of random 
variables. O ur a pproach follows the work of LaurentI ( 19961 . l2005l ). and we also refer to 
LevitI ( 19781 ) and iKerkvacharian and Picardl ( 19961 ) for general results on nonlinear func- 



tionals estimation. Such functionals of a density appear in i nany statistical applications 



and their efficient estimation remains an active research field (Chacon and Tenreiro 2011 



Gine and Nicklll2008l : lGine and Masonll2008l ). However we consider functionals involving 



conditional densities, which necessitate a specific treatment. The estimate obtained here 
can be used for global SA involving general conditional moments, but it includes as a 
special case Sobol sensitivity indices. Note also that an extension of the approach devel- 
oped in our work is simultaneously proposed in the context of sliced inverse regression 
(|Loubes et al.ll201ll ). 



February 26, 2013 21:14 Journal of Nonparametric Statistics EfficicntSA'ncwvcrsion 



The paper is organized as follows. Section [2] first recaps variance-based methods for 
global SA. In particular, we point out which type of nonlinear functional appears in 
sensitivity indices. Section [3] then describes the theoretical framework and the proposed 
methodology for building an asymptotically efficient estimator. In Section HI we focus 
on Sobol sensitivity indices and study numerical examples showing the good behavior of 
the proposed estimate. We also illustrate its interest on a reservoir engineering example, 
where uncertainties on the geology propagate to the potential oil recovery of a reservoir. 
Finally, all proofs are postponed to the appendix. 



2. Global sensitivity analysis 

In many applied fields, physicists and engineers are faced with the problem of estimating 
some sensitivity indices. These indices quantify the impact of some input variables on an 
output. The general situation may be formalized as follows. 

The output y S M is a nonlinear regression of input variables r = (ri, . . . ,r;) (Z > 1 is 
generally large). This means that Y and r satisfy the input-output relationship 

Y = cl>(r) (1) 

where $ is a known nonlinear function. Usually, $ is com plicated and has not a close d 
form, but it may be computed through a computer code ( Qaklev and O'Haganll2004l ). 



In general, the input r is modelled by a random vector, so that Y is also a random 
variable. A common way to quantify the impact of input variables is to use the so-called 
Sobol sensitivity indices ( Sobol1ll993l ). Assuming that all the random variables are square 



integrable, the Sobol index for the input Tj {j = 1, . . . ,1) is 

^ Var(E(r|r,))^ ^^ 



Observing an i.i.d. sample (Yi, r^^^), . . . , {Yn, r^")) (with Yi = <^(tW), i = 1, . . . ,n), the 
goal is is then to estimate Sj (j = 1, . . . , /). Obviously, ([2]) may be rewritten as 

E(E(y|r,)^)-E(y)^ 
^ Var(y) 

Thus, in order to estimate Ej, the hard part is E(E(y|rj)^). In this paper we will provide 
an asymptotically efficient estimate for this kind of quantity. More precisely we will tackle 
the problem of asymptotically efficient estimation of some general nonlinear functional. 
Let us specify the functional we are interested in. Let {Yi,Xi), . . . , (y„, X„) be a sample 
of i.i.d. random vectors of M? having a regular density / (see Section [3] for the precise 
frame) . We will study the estimation of the nonlinear functional 



r(/) = E(v(E(v.(y)|x))) 



^'i^5f^'^'-'""^ 



February 26, 2013 21:14 Journal of Nonparametric Statistics EfficicntSA'ncwvcrsion 



where ip ^-^d (p are regular functions. Hence, the Sobol indices are the particular case 
obtained with ■(/'(C) = C^ and ip{^) = ^. 

The method developed in o rder to obtain an asymptotically efficient estimate for T(/) 
follows the one developed by lLaurentl ( 19961 ). Roughly speaking, it involves a preliminary 



estimate / of / built on a small part of the sample. This preliminary estimate is used in a 
Taylor expansion of T(/) up to the second order in a neighbourhood of /. This expansion 
allows to remove the bias that occurs when using a direct plug-in method. Hence, the 
bias correction involves a quadratic functional of /. Due to the form of T, this quadratic 
functional of / may be written as 



0{f) = /// ri{x,yi,y2)f{x,yi)f{x,y2)dxdyidy2. 
This kind of functio nal does not fall in the frame treated in iLaurentl ( 19961 ) or 



Gine and Nickll ( 20081 ) and have not been studied to the best of our knowledge. We 



study this problem in Section 13.11 where we build an asymptotically efficient estimate for 
0. Efficient estimation of T{f) is then investigated in Section | 



3. Model frame and method 

Let a < b and c < d, Lp'{dxdy) will denote the set of square integrable functions on 
[a, 6] X [c, d]. Further, L?'{dx) (resp. L'^{dy)) will denote the set of square integrable 
functions on [a, 6] (resp. [c, d]). For sake of simplicity, we work in the whole paper with 
the Lebesgue measure as reference measure. Nevertheless, most of the results presented 
can be obtained for a general reference measure on [a, h] x [c, d\. Let (aj^(x))j^gi:)j (resp. 
i(3if,{y))if,eD2) be a countable orthonormal basis of L'^{dx) (resp. of Lp'{dy)). We set 
Pi{x,y) = ai^{x)f3i^{y) with i = {ia,ii3) £ D := Di x D2. Obviously {pi{x,y))i(,D is a 
countable orthonormal (tensor) basis of L'^{dxdy). We will also use the following subset 
of L'^{dxdy) : 



S = { y^ GiPi : {ei)i^D is a sequence with N^ 

KieD i&D 



2 



here {ci)i^D is a given fixed positive sequence. 

Let (X, Y) having a bounded joint density / on [a, b] x [c, d\ from which we have a sample 
{Xi,Yi)i=i^,„^n- We will also assume that / lies in the ellipsoid £. Recall that we wish to 
estimate a conditional functional 



E 



(^(E(<^(y)|x)) 



where (^ is a measurable bounded function with Xi ^ V' ^ X2 and ijj G C''^([xi, X2]) the 
set of thrice continuously differentiable functions on [xi;X2]- This last quantity can be 
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expressed in terms of an integral depending on the joint density /: 



-'^>-//H'^5m^l/(-.)<^* 



ip{m{x))f{x, y)dxdy 



where m{x) = J (p{y)f{x,y)dy/ J f{x,y)dy is the conditional expectation of c^(y) given 
{X = x). We suggest as a first step to consider a preliminary estimator / of /, and to 
expand T(/) in a neighborhood of /. To achieve this goal we first define F : [0, 1] — )■ M : 

F{u)=T{uf+{l-u)f) {ue[0,l]). 
The Taylor expansion of F between and 1 up to the third order is 

F(l) = F(0) + F'(0) + ^F"(0) + ^F'"(0(1 - 0' (3) 



for some ^ €]0, 1[. Here, we have 



F(i) = r(/) 



and 



F(0) = T(/) = / / ^. i^Ml^I^j /(., y),:,iy 

■ilj{m{x))f{x, y)dxdy 



where rh{x) = J (p{y)f{x,y)dy/ J f{x,y)dy. Straightforward calculations also give 
higher-order derivatives of F : 

F'(0) = // ([(p{y) -m{x)]ij{m{x))+4j{m{x))) (f{x,y)- f{x,y))dxdy 



F"{0) =111 ^i^^^^^(rn(x) - ^{y)) (m(x) - ^{z)) 

J f{x,y)dy 



fix, y) - fix, y)j \^f{x, z) - f{x, z)j dxdydz 
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F"'{0 




f{x,y)dy 



a{x,y) + {l-i)f{x,y)dy'^ 
[{m{x) - ip{y)) [m{x) - ipiz)) [m{x) - ip{t)) 

f{x, y)dy I 'ii> (f(^, x)) - 3(m(x) - ip{y)) (m(x) 



^(^)) 



[U{x, y) + (1 - i)f{x, y)]dy i, (f(e, x)) 



f{x, y) - f{x, y)j [^f{x, z) - f{x, z) 
f{x, t) — f{x, t) ) dxdydzdt 



where f{^,x) 



Jip{ymix,y) + il-Ofix,y)]dy 



J[U{x,y) + {l-Ofix,y)]dy 
first derivatives of "ip. 



and tp, ip and ip denote the three 



Plugging these expressions into ([3|) yields the following expansion for T{f): 



nf) 



+ 



H{f,x,y)f{x,y)dxdy 



K{f, X, y, z)f{x, y)f{x, z)dxdydz + r„ 



(4) 



where 



H{f, X, y) = [p{y) - m(x)] '4){m{x)) + i/j{rh{x)), 
1 ip{m{x)) 



K{f,x,y,z) 



f{x,y)dy 



[rn{x) - ip{y)) {m{x) - ip{z)), 



1 





for some ^ g]0, 1[. Notice that the first term is a linear functional of the density /, it will 
be estimated with 



-^H{f,X„Y,). 

"-2 ^ 



The second one involves a crossed term integral which can be written as 



vix, yi,y2)fix, yi)f{x, y2)dxdyidy2 



(5) 



where ry : R^ — )• M is a bounded function verifying r/(x, 2/1,2/2) = v{x:y2:yi) for all 
{x, 2/1,^/2) £ R^- In summary, the first term can be easily estimated, unlike the second one 
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which deserves a specific study. In the next section we then focus on the asymptotically 
efficient estimation of such crossed quadratic functionals. In Section [3.21 these results are 
finally used to propose an asymptotically efficient estimator for T(f). 



3.1. Efficient estimation of quadratic functionals 

In this section, our aim is to build an asymptotically efficient estimate for 



vix, yi,y2)fix, yi)fix, y2)dxdyidy2. 

We denote ai = j fpi the scalar product of / with pi as defined at the beginning of 
Section [31 We will first build a projection estimator achieving a bias equal to 

[SMf{x,yi) - f{x,yi)] [SMf{x,y2) - f{x,y2)]r]{x,yi,y2)dxdyidy2 

where SmJ = SieA/ ^*P« ^^^ M is a subset of D. Thus, the bias would only be due to 
projection. Developing the previous expression leads to a goal bias equal to 

SMf{x, yi)f{x, y2)v{x, yi,y2)dxdyidy2 

SaiHx, yi)SMf{x, y2)v{x, yi,y2)dxdyidy2 

f{x, yi)f{x, y2)v{x, yi,y2)dxdyidy2. (6) 

Consider now the estimator 9n defined by 
c^ 2 




^ XI XI Pi{Xj,Yj) pi{Xk,u)r]{Xk,u,Yk)du 



n(n 



1 " 

-- X E MX„Y,)pAXk,Yk) 



nin — 1) ^ 

^ ' i,i'£M j=^k=l 

Pi{x, yi)Pi'{x, y2)'n{x, yi,y2)dxdyidy2. (7) 

This estimator achieves the desired bias : 

Lemma 3.1: The estimator On defined in ^ estimates 9 with bias equal to 



[SMf{x,yi) - f{x,yi)][SMfix,y2) - f{x,y2)]ri{x,yi,y2)dxdyidy2. 



Since we will carry out an asymptotic analysis, we will work with a sequence {Mn)n>i 
of subsets of D. We will need an extra assumption concerning this sequence: 
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Al. For all n > 1, we can find a subset M„ C D such that (supjrfj^,^^ 



\M„ 



n2 (A 



Bn means Ai < A^/Bn < A2 for some positive constants Ai and A2)- Furthermore, 
Vt G L'^{dxdy), / {SM^t — t) dxdy — )• when n — )• 00. 

The following theorem gives the most important properties of our estimate On ■ 
Theorem 3.2 : Assume Al hold. Then On has the following properties: 
(i) If \Mn\/n — )■ when n — t- 00, then 



niOn-0] ^AA(0,A(/,r?)), 



(8) 



A(/,^) 



< 71 



|M„, 



n 



+ \\SMj-f\\2 + \\SM^g-gh 



(9) 



where g{x,y) := / f{x,u)ri{x,y,u)du and 



Hf,r])=4: 



g{x,yff{x,y)dxdy 



(a) Otherwise 



E 



[on-oy 



g{x,y)f{x,y)dxdy 



< 



\Mn 



72- 



n 



where 71 and'j2 <ire constants depending only on ||/||oo; Halloo and Ay (with Ay = d — c). 
Moreover, these constants are increasing functions of these quantities. 

Remark 1 : Since in our main result (to be given in the next section) r] will depend on 
n through the preliminary estimator /, we need in ([9]) a bound that depends explicitly 
on n. Note however that Q implies 



lim nE{0n-0) = A{f,r]). 



The asymptotic properties of On are of particular importance, in the sense that they 
are optimal as stated in the following theorem. 

Theorem 3.3 : Consider the estimation of 



Oil) 




V{x, yi , y2)f{x, yi)f{x, y2)dxdyidy2. 



Let /o G £. Then, for all estimator On of 0{f) and every family V(/o) of vicinities of /o, 
we have 

inf liminf sup nE(^„ - ^(/o))^ > A(/o,r?). 
{v(/o)} "^00 /ev{/o) 



February 26, 2013 21:14 Journal of Nonparametric Statistics EfficicntSA'ncwvcrsion 



In other words, the optimal asymptotic variance for the estimation of 9 is A(/o, rj). As 
our estimator defined in ([7]) achieves this variance, it is therefore asymptotically efficient. 
We are now ready to use this result to propose an efficient estimator of T{f). 



3.2. Main Theorem 

In this section we come back to our main problem of the asymptotically efficient estima- 
tion of 

Recall that we have derived in Q an expansion for T{f). The key idea is to use here 
the previous results on the estimation of crossed quadratic functionals. Indeed we have 
provided an asymptotically efficient estimator for the second term of this expansion, 
conditionally on /. A natural and straightforward estimator for T(/) is then 



-1 '"Z 

•nr, ^—^^ 



2 



^ "2 

Pi{x, yi)pi'{x, y2)K{f, x, yi,y2)dxdyidy2. 



In the above expression, one can note that the remainder r„ does not appear : we will see 
in the proof of the following theorem that it is negligible comparing to the two first terms. 

In order to study the asymptotic properties of T„, some assumptions are required 
concerning the behavior of the joint density / and its preliminary estimator / : 

A2. supp/ C [a,b] x [c,d] and V(x,y) G supp/, < a < f{x,y) < (3 with q,/3 G M 

A3. One can find an estimator f of f built with ni « n/log(n) observations, such that 

V(x, y) G supp/, < a - e < f{x, y) < 13 + e. 
Moreover, 

V2 < g < +00, V/ G N*, ^f\\f-f\\\ < C{q,lW^ 

for some A > 1/6 and some constant C(g, /) not depending on / belonging to the 
ellipsoid £. 
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Here supp/ denotes the set where / is different from 0. Assumption A2 is restrictive 
in the sense that only densities with compact support can be considered, excluding for 
example a Gaussian joint distribution. 

Assumption A3 imposes to the estimator / a convergence fast enough towards /. We 
will use this result to control the remainder term r„. 

We can now state the main theorem of the paper. It investigates the asymptotic prop- 
erties of Tn under assumptions Al, A2 and A3. 

Theorem 3.4 : Assume that Al, A2 and A3 hold. Then Tn has the following properties 

. \Mn\ 

if J '- -^ 0: 

n 

V^(fn-T{f))^M{0,C{f)), (10) 



limnE(f„-r(/))' = C7(/), (11) 

where C{f) = E(var{ip{Y)\X) ij{E{Y\X)) J + VarU{E{ip{Y)\X))Y 

We can also compute as in the previous section the semiparametric Cramer-Rao bound 
for this problem. 

Theorem 3.5 : Consider the estimation of 

^^^^ = lh{ ^ff{x^t' ) ^(^'^^^^'^^ = ^(^(^(^(ni^))) 

for a random vector {X,Y) with joint density f €z £ . Let fo^Sbea density verifying 
the assumptions of Theorem \3.4\ Then, for all estimator T„ of T{f) and every family 
V(/o) of vicinities of fo, we have 

inf liminf sup nE(Tn - T {fo))^ > C {fo) . 

{V(/o)} n^oo /gv(/o) 

Combination of theorems 13.41 and 13.51 finally proves that T„ is asymptotically efficient. 



4. Application to the estimation of sensitivity indices 

Now that we have built an asymptotically efficient estimate for T{f), we can apply it 
to the particular case we were initially interested it: the estimation of Sobol sensitivity 
indices. Let us then come back to model ([T|) : 

Y = $(t) 
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where we wish to estimate ([2]): 



_ Var(E(y|rj)) _ E{E{Y\Tjf) -E{Yf 
^^ ~ Var(y) ~ Var(y) 



To do so, we have an i.i.d. sample {Yi,t^^>), . . . , (y„,T*-"'-*). We will only give here the 
procedure for the estimation of Si since it will be the same for the other sensitivity 
indices. Denoting X := ti, this problem is equivalent to estimating E(E(y|X)^) with an 
i.i.d. sample {Yi,Xi), . . . , (y„, X„) with joint density /. We can hence apply the estimate 
we developed previously by letting ^p{^) = ^^ and ip{^) = ^: 

r(/) = E(E(y|x)2) 

fff jyf{x,y)dy \\^ 
= ll\!f{x,y)dy) /(-'^)^-^^- 

The Taylor expansion in this case becomes 

T{f) = jj H{f,x,y)f{x,y)dxdy 

+ K{f,x,y,z)f{x,y)f{x,z)dxdydz + Tn 

where 

H{f, X, y) = 2ym{x) - m{xf, 

K{f, X, y, z) = - — ; ^ (m(x) - y) (m(x) - z) 

f f{x,y)dy) 



and the corresponding estimator is 

f„ = — VF(/,x,-,y,) 

2 



^2 . _, 



+ 



"'•2 p 

nE E p^iXJ,YJ) p,iXk,u)Kif,Xk,u,Yk)dz 



Mn2 ^, ,^M,^k=i 



-. "2 

Pi{x, yi)pi'{x, y2)K{f, x, yi,y2)dxdyidy2. 



for some preliminary estimator / of /, an orthonormal basis {pi)i£D of Lp'{dxdy) and a 
subset M G D verifying the hypotheses of Theorem 13.41 



We propose now to investigate the practical behavior of this estimator on two analyti- 
cal models and on a reservoir engineering test case. In all subsequent simulation studies, 
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Table 1. Conditional moments for analytical model (I12|l . Mean and standrad 
deviation of T„ for different values of n. 



Inputs 


E{E{Y\T,f) 


n = 100 


T 
n = 10000 


Configuration (a) 

n 

T2 


0.5733 
0.5611 


0.5894 +/- 0.052 
0.5468 +/- 0.054 


0.5729 +/- 0.005 
0.5611 +/- 0.005 


Configuration (b) 

n 

T2 


314.04 
779.85 


305.98 +/- 52.1 
814.04 +/- 10.3 


318.27 +/- 7.52 
787.82 +/- 0.53 


Configuration (c) 

n 

T2 


16258 
44034 


18414 +/- 3759 
44667 +/- 82.6 


16897 +/- 427 
44073 +/- 8.17 



the preliminary estimator / will be a kernel density estimator with bounded support 
built on ni = [log(n)/n] observations. Moreover, we choose the Legendre polynomials 
on [a,h] and [c, d] to build the orthonormal basis {pi)i^D and we will take \M\ = y/n. 
Finally, the integrals in T„ are computed with an adaptive Simpson quadrature. 



4.1. Simulation study on analytical functions 

The first model we investigate is 



Y 



ri+rl 



(12) 



where three configurations are considered (ti and T2 being independent): 



(a) 
(b) 
(c) 



Z^(0,l),i = l,2; 
U{0,5),j = l,2. 



For each configuration, we report the results obtained with n = 100 and n = 10000 in 
Table [H Note that we repeat the estimation 100 times with different different random 
samples of {ti,T2). 

The asymptotically efficient estimator T„ gives a very accurate approximation of 
sensitivity indices when n = 10000. But surprisingly, it also gives a reasonably accurate 
estimate when n only equals 100, whereas it has been built to achieve the best symptotic 
rate of convergence. 

It is then interesting to compare it with other estimators, more precisely two nonpara- 
metric estimators that have been specifically built to give an accurate approximation 
of sensitivit y indices when n is not larg e. The first one is based on a Gaussian process 
metamodel (lOaklev and O 'Hagan 20041 ) . while the other one involves local polynomial 



estimators (jDa Veiga et al.i.2009. ). The comparison is performed on the following model 
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Table 2. Comparison between efficient estimation and nonparametric estimates 



on analytical model (|13|. 








True value 


Oakley-O'Hagan 


Local polynomials 


f„ 


Var{E{Y\X^)) 1.0932 
Var(E(yx2)) 0.0729 


1.0539 
0.1121 


1.0643 
0.0527 


1.1701 
0.0939 



Y = 0.2exp(ri - 3) + 2.2\t2\ + 1.3r| - 2t| - 0.5t| - O.Srf 

+2.5rf + O.rrf + (^^^ _ ,), ^ ^^^ _ 3), ^ ^ + sin(5n) cos(3rf ) (13) 

where ri and r2 are independent and uniformly distributed on [—1,1]. This nonhnear 
function is interesting since it presents a peak and vaheys. We estimate the sensitivity 
indices with a sample of size n = 100, the results are given in Table El 

Globally, the best estimates are given by the local polynomials technique. However, 
the accuracy of the asymptotically efficient estimator T„ is comparable to that of the 
nonparametric ones. These results confirm that r„ is a valuable estimator even with a 
rather complex model and a small sample size (recall that here n = 100). 



4.2. Reservoir engineering example 

The PUNQ test case (Production forecasting w ith UNcertainty Qua ntification) is an 



oil reservoir model derived from real field data ( Manceau et al.ll200ll ). The considered 
reservoir is surrounded by an aquifer in the north and the west, and delimited by a fault 
in the south and the east. The geological model is composed of five independent layers, 
three of good quality and two of poorer quality. Six producer wells (PRO-1, PRO-4, PRO- 
5, PRO-11, PRO-12 and PRO-15) have been drilled, and production is supported by four 
additional wells injecting water (XI, X2, X3 and X4). The geometry of the reservoir and 
the well locations are given in Figure [U left. 

In this setting, 7 variables which are characteristic of media, rocks, fluids or aquifer 
activity, are considered as uncertain: the coefficient of aquifer strength (AQUI), 
horizontal and vertical permeability multipliers in good layers (MPVl and MPHl, 
respectively), horizontal and vertical permeability multipliers in poor layers (MPV2 
and MPH2, respectively), residual oil saturation after waterflood and after gas flood 
(SORW and SORG, respectively). We focus here on the cumulative production of oil of 
this held during 12 years. In practice, a fluid flow simulator is used to forecast this oil 
production for every value of the uncertain parameters we might want to investigate. 
The uncertain parameters are assumed to be uniformly distributed, with ranges given in 
Table [31 We draw a random sample of size n = 200 of these 7 parameters, and perform 
the corresponding fluid-flow simulations to compute the cumulative oil production after 
12 years. The histogram of the production obtained with this sampling is depicted in 
Figure [H right. Clearly, the impact of the uncertain parameters on oil production is 
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Figure 1. Left: top view of the PUNQ reservoir. Producer and injector wells are indicated by 
black and white circles, respectively. OWC and GOW stand for Oil Water Contact and Gas Oil 
Contact. Right: histogram of the cumulative production after 12 years (10^ m^). 



Table 3. Range of variation and estimated first-order sensitivity 
index of the uncertain parameters of the PUNQ model. 

Parameter Range of variation Estimated sensitivity index 

with f„ (%) 



AQUI 


0.2 - 0.3 


7.206 


MPHl 


0.8- 1.2 


40.929 


MPH2 


0.8- 1.2 


0.419 


MP VI 


0.8- 1.2 


0.041 


MPV2 


0.8- 1.2 


0.693 


SORG 


0.15 - 0.2 


0.338 


SORW 


0.15 - 0.25 


49.4 



large, since different values yield forecats varying by tens of thousands of oil barrels. 
In this context, reservoir engineers aim at identifying which parameters affect the most 
the production. This help them design strategies in order to reduce the most influential 
uncertainties, which will reduce, by propagation, the uncertainty on production forecasts. 



In this context, computation of sensivity indices is of great interest. Starting from 
the random sample of size n = 200, we then estimate the first-order sensitivity index 
of each parameter with the estimator T„. Results are given in Table [3l As expected, 
the most infiuential parameters are the horizontal permeability multiplier in the good 
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reservoir units MPHl and the residual oil saturation after waterflood SORW. Indeed, fluid 
dispacement towards the producer wells is mainly driven by the permeability in units 
with good petrophysical properties and by water injection. More interestingly, vertical 
permeability multipliers do not seem to impact oil production in this case. This means 
that fluid displacements are mainly horizontal in this reservoir. 



5. Discussion and conclusions 

In this paper, we developed a framework to build an asymptotically efficient estimate 
for nonlinear conditional functionals. This estimator is both practically computable and 
has optimal asymptotic properties. In particular, we show how Sobol sensitivty indices 
appear as a special case of our estimator. We investigate its practical behavior on two an- 
alytical functions, and illustrate that it can compete with metamodel-based estimators. A 
reservoir engineering application case is also studied, where geological and petrophysical 
uncertain parameters affect the forecasts on oil production. The methodology developed 
here will be extended to other problems in forthcoming work. A very attractive extension 



i s the construction of an adaptive procedure to calibrate the size of M„ as done in lLaurent 



( 2OO5I ) for the L^ norm. However, this problem is non obvious since it would involve 



treati ng refined inequalities on U-statistics such as presented in iHoudret and Revnaud 



( 2OO2I ). From a sensitivity analysis perspective, we will also investigate efficient estima- 



tion of other indices based on entropy or other norms. Ideally, this would give a general 
framework for building estimates in global sensitivity analysis. 
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Appendix A. Proofs of Theorems 

A.l. Proof of Lemma \3.1\ 

Let 9n = 0^- ^n where 



n { 






and 



^'- ^E E ^'(^i'^i)^^'(^fc'^fc 



n(n — 1) ^ 

^ ' i,i'£M j^k=l 

/ Pi{x,yi)pi'{x,y2)v{x,yi,y2)d. 



'xdyidy2 
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Let us first compute IE(^^) : 

lE(^n) = 2^ // Piix,y)f{x,y)dxdy /// pi{x,y)r]{x,u,y)f{x,y)dxdydi 

= 2^ai / // pi{x,y)r]{x,u,y)f{x,y)dxdydu 



ieM 



^ aiPi{x,y) I r]{x,u,y)f{x,y)dxdydi 

Shifix, y)r]{x, u, y)fix, y)dxdydu. 
J J J 

Furthermore, 

^^01)= XI Pii^^y)fi^^y)d^dy pi>{x,y)f{x,y)dxdy 

Pi{x, yi)pi'{x, y2)r}{x, yi,y2)dxdyidy2 



X aiUi' Pi{x,yi)pi'{x,y2)v{x,yi,y2)dxdyidy2 

/ I XI "-iPii^^yi) ) I X "-i'Pi'i^^y-^) ) r]ix,yi,y2)dxdyidy2 

•^ \ieM J Vi'GAf / 

SmKx, yi)SMf{x, y2)v{x, yi,y2)dxdyidy2. 
Finally, E(^„) - 6* = E(^i) - E(i92) - 6 and we get the desired bias with dSD. 

A. 2. Proof of Theorem \3IE 

We will write M instead of M„ for readability and denote m = \M\. We want to bound 
the precision of 9n- We first write 



^{On- / / / ■nix,yi,y2)fix,yi)f{x,y2)dxdyidy2 = Bias^(0„) + Var(^n) 




The first term of this decomposition can be easily bounded, since 6n has been built to 
achieve a bias equal to 



Bias(6i„) = - [SMf{x,yi) - f{x,yi)][SMf{x,y2) - fix,y2)] 

V{x,yi,y2)dxdyidy2. 
We then get the following lemma : 
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Lemma A.l: Assuming the hypotheses of Theorem \3.S\ hold, we have 



Proof: 



\Bias{9n)\ < Ay||ry||oo sup |cip. 



|Bias((9n)| < ||r/||oo / ( / \SMf{x,yi) - f{x,yi)\dy-i 
\SMfix,y2) - f{x,y2)\dy2 I dx 



2 



< Ihlloo / ( / \SMf{x,y) - f{x,y)\dy\ dx 

< Ay||r/||oo / / {SMf{x,y) - f{x,y)fdxdy 

< Ay||r/||oo ^ |aj|^ < Ay||r/||oo sup \ci 



HM 



|2 

-J I 

tM 



Indeed, f £ £ and the last inequality follows from Holder inequality. D 

Bounding the variance of On is however less straightforward. Let A and B be the mxl 
vectors with components 



fli := / / fix, y)piix, y)dxdy i = l,... ,m 
bi ■■= II Piix,yi)f{x,y2)r}{x,yi,y2)dxdyidy2 
g{x,y)pi{x,y)dxdy i = l,...,m 



where g{x, y) = f{x, u)rj{x, y, u)du for each i G M. a, et bi are the components of / and 
g onto the ith component of the basis. Let Q and R be the mxl vectors of the centered 
functions qi{x,y) = pi{x,y)—ai and ri{x,y) = / pi{x , u)ri{x , u, y)du — biiov i = 1, . . . ,m. 



Let C be the mxm matrix of constants cw = / / / Pi{x, yi)pi'{x, y2)rj{x, yi,y2)dxdyidy2 

for i,i' = l,...,m. Take care that here cu' is double subscript unlike in the 
(cj) sequence appearing in the definition of the ellipsoid £. We denote by [/„ 

1 " 

the process Unh = — > h{Xj,Yj,Xk,Yk) and by Pn the empirical measure 

nln — 1) ^-^ 

1 " 
Pnf = — 2_. fi-^j^^j)- With the previous notation, 9n has the following Hoeffding's de- 

j=i 

composition (see chapter 11 of I Van Per Vaartl ( 19981 )): 



9„ = UnK + PnL + 2^AB - ^ACA (Al) 
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where 



K{xi,yi,X2,y2) = 2*Q{xi,yi)R{x2,y2) - *Q{xi,yi)CQ{x2,y2), 

L{xi,yi) = 2'AR{xi,yi) + 2*5Q(xi, yi) - 2*ACQ(xi, yi). 



Then Var(^„) = Yav{UnK) + Var(P„L) + 2 Cov([/„K, P„L). We have to get bounds for 
each of these terms : they are given in the three following lemmas. 

Lemma A. 2: AssuTning the hypotheses of Theorem \3.2\ hold, we have 

20 

Var{UnK) < -^-_^||r?||^||/||^A^(m + 1). 



Proof: Since UnK is centered, Yar (UnK) equals 



E 
1 






n{n — 1) 



EiK\Xi,Yi,X2,Y2) + K{Xi,Yi,X2,Y2)K{X2,Y2,Xi,Yi)). 



By the Cauchy-Schwarz inequality, 



Yai{UnK) < —^—E{K\Xi,Yi,X2,Y2)). 
n{n — 1) 



Moreover, the inequality 2|E(Xy)| < E{X'^) + E{Y'^) leads to 

E{K^{Xi,Yi,X2, Y2)) < 2 [E {{2Q'{Xi,Yi)R{X2, ^2))') 

+E((Q'(Xi,yi)CQ(X2,l2))')] 

We have to bound these two terms. The first one is 



{{2Q'{Xi,Yi)R{X2, Y2)f) = A{Wi - VF2 - T^s + W^) 
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where 

Wi= jjjjjj '^Piix,y)Pi'ix,y)pi{x',u)pi'{x',v7]{x',u,y)7]{x',v,y') 
f{x, y)f{x', y')dudvdxdydx' dy 



Wi= \\ '^bibi'Pi{x,y)pi,{x,y)f{x,y)dxdy 

W3= //// '^aiai'Pi{x,u)pi'{x,v)r]{x,u,y)r]{x,v,y)f{x,y)dxdy 
Wi = y^^aiai'bjbr 



•Ji'- 

i.i' 



Straightforward manipulations show that W2 > and VF3 > 0. This imphes that 

E {{2Q'{Xi,Yi)R{X2, Y2)f) < 4{Wi + W4). 
On the one hand, 




<„.„2 




00 



Wi = / // / ^Pi{x,y)Pi'{x,y) / pi{x',u)r]{x',u,y')du / pi,{x',v)'rj{x',v,y')dvf{x,y)f{x',y')dxdydx'dy' 

- //// I XlP«(^'y) / ^i(^''")^(^''^'2^')'^^^| f{x,y)f{x',y')dxdydx'dy' 

< II fllL / / / / I y^Pi{x,y) / pi{x',u)ri{x' ,u,y')du dxdydx'dy' 
^Pi{x,y)pi'{x,y) / pi{x',u)ri{x',u,y')du / pi'{x' ,v)r]{x' ,v,y')dvdxdydx'dy' 

< II/II»E// Upiix',nHx',u,y')du\ dx'dy' 

since the pi are orthonormal. Moreover, 

< ll^llL^r / Piix',ufdu, 
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and then 



Pi{x' ,u)rj{x' ,u,y')du\ dx'dy' < \\r]\\^A.Y pi{x',u) dudx 



Finally, 



W^<Mi,\\f\\t,l^i^m 



On the other hand, 



w, = YaA <yAyA< mlMl < ii/iiooNii. 



By the Cauchy-Scharwz inequality we have \\g\\2 < ll'?llooll/llooAy and then 

W4<\\v\L\\f\\l^Y 

which leads to 

E {{2Q'iX,,Yi)RiX2, Y2)f) < 4||r/||^||/||^A|-(m + 1). 
Let us bound now the second term E {{Q'{Xi,Yi)CQ{X2, ¥2))^) = W5-2W6 + W7 where 

W5 = /// / yycii,Ci^i'^pi{x,y)pi,{x,y)pi>{x',y')pi'^{x,y)f{x,y)f{x',y')dxdydxdy' 

W^6 = ^^ // Cii'Ci^i'^aiai,pi'{x,y)pi'^{x,y)f{x,y)dxdy 
Wy = 'yycii'Ci^i'^aitti^ai'ai'^. 

Following the previous manipulations, we show that Wq > 0. Thus, 

E{{Q'{X,,Yi)CQ{X2,Y2)f) <W^ + W-j. 
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First, observe that 



^^= \^Cii'Pi{x,y)pi'{x',y')\ f{x,y)f{x',y')dxdydx'dy' 



< „^,|2 



// \^'^ii'Pii^^y)Pi'i^''y') \ dxdydx'dy' 



< WfWloYlYl^ii'^i^^'i j j j j Pii^^y)PiA^^y) 

Pi' {x', y')pi'^ {x', y')dxdydx'dy' 



< „^,|2 



since the pi are orthonormal. Besides, 



^'^u' = // Xl"*°(^)"*i(^)"*°(^')"*i(^')Xl ( // /5«3(yi)A^ (2/2)^(2;, yi, 2/2)^2/1% 
liif>{yi)Pidy2)'n{x',yi,y2)dyidy2 I dxdx' 






A^ (yi) A^ {y2)r]{x, yi , y2)dyidy^^ 



(iip{yi)Pi'^{y2)'n{x',yi,y2)dyidy2 ) dxdx'. 



But 



XI ( // A;3(yi)A3(y2)^(a;,yi,y2)rfyirfy2 



ft^ (yi)ft^ {y2)v{x', yi,y2)dyidy2 

Y^ / / / / ft^ (yi)A^ {y2)ri{x, yi,y2)(3if, (yi)A^ iy'2)v{x', y[,y2)dyidy2dy[dy2 
l^i l^ifAyi)v{x-,yi-,y2)dyi\ /3j^(yi)X ( / (3i'^iy'2)v{x',y'i,y2)dy'2J I3i'^{y2)dy[dy2 



V{x, y'l , y2)v{x', y[,y2)dy'idy2 



.2 11^112 

loo 



February 26, 2013 21:14 Journal of Nonpararnctric Statistics EfficicntSA'ncwvcrsion 

24 REFERENCES 



using the fact that (/3j) is an orthonorinal basis. We then get 



i,i' 



<A|-||r? 
<A|-||r? 

<A|-||r? 



I I Z^ *^*° (^)'^«L (^)'^ia {^')'^i'^ {x')dxdx' 
y I / ai^{x)ai'^{x)dx 

y^ f / ai^{xfdx] 



-,m 



since the ai are orthonornial. Finally, 



W^5 < ||??| 



1/^lm. 



Besides, 



^r= E 



Cii' didi' 



with 



/ ^ Cii'CLiCli' 



— 1 1 '/ 1 1 CO 

— Il'/lloo 



|5'm/(x, yi)SMfix, y2)\dxdyidy2 
\SMfix,yi)SMfix,y2)\dx] dyidy2- 
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By using the Cauchy-Schwarz inequality twice, we get 



'^Cii'ttiai, < Ay 



<Af. 



<A^ 



< A^ 



<A2, 



\SMf{x,yi)SMf{x,y2)\dx j dyidy2 
Shifiu^yifdu] ( / SMf{v,y2fdv ] dyidy2 




SMf{u,yi) SMf{v,y2) dudvdyidy2 

^ 2 

Sufix^yfdxdy 



2 

oo' 



Finally, 



E((Q'(Xi,yi)CQ(X2,y2))' 



< 



2 a2 



loo lu Moo 



Af(m + 1) 



Collecting this inequalities, we obtain 



\&T{UnK) < 



20 



n{n — 1) 



MlLll/foo^' 



A|.(m + 1) 



which concludes the proof of Lemma IA.2[ 



D 



Let us now deal with the second term of the Hoeffding's decomposition of 9r, 



Lemma A. 3: Assuming the hypotheses of Theorem \3.2\ hold, we have 



36 



Var{P,L)<-Al\\f\\lMl. 



n 



Proof: First note that 



Var(P„L) = -Var(L(Xi,yi)). 



February 26, 2013 21:14 Journal of Nonpararnctric Statistics EfficicntSA'ncwvcrsion 

26 REFERENCES 



We can write L{Xi,Yi) as 

L{Xi,Yi) = 2A'R{Xi,Yi) + 2B'Q{Xi,Yi) - 2A'CQ{Xi,Yi) 
= 2^ai f / pi{Xi,u)ri{Xi,u,Yi)du - bA 

+2^5i(pi(Xi,yi) - ai) -2'^Cii.ai\pi{Xi,Y{) - m) 
= 2 /" ^ aiPi{Xuu)r]{Xi,u, Yi)du + 2Y,hVi{Xi,Y{) 

i i 

~2Y,Cn'ai.p^{Xi,Yi) - AA'B + 2^CA 

= 2 j SMf{Xi,u)r]{Xi,u, Yi)du + 2Sm9{XuYi) 
-2Y,Cn'ai.p^{Xi,Yi) - AA'B + 2A'CA. 

Let h{x,y) = / SMf{x,u)ri{x,u,y)du, we have 

" XI ( /// SMfix,u)r]{x,u,y)pi{x,y)dudxdy] pi{z,t) 



y^ f /// ai'Pi'{x,u)r]{x,u,y)pi{x,y)dudxdy]pi{z,t) 
^Cii'ai'Pi{z,t) 



and we can write 

L{Xi,Yi) = 2h{Xi,Yi) + 2Sm9{XuYi) - 2SMh{Xi,Yi) - AA'B + 2A'CA. 
Thus, 

Var(L(Xi,yi)) = 4Var[/i(Xi,yi) + Sm9{Xi,Yi) - SMh{Xi,Yi)] 



< AE[{h{Xi,Yi) + SMgiXi,Yi) - SMhiXi,Yi)f] 

< 12E[{h{Xi,Yi)f + {SM9{XuYi)f + {SMh{Xi,Yi)f] 
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Each of these three terms has to be bounded 



Ei{hiXi,Y,)f) 



SmKx, u)r]{x, u, y)du f{x, y)dxdy 



<Ay /// SMf{x,ufr]{x,u,yff{x,y)dxdydu 



<Af.||/||oo 


I^IlL // SMf{x,ufdxdu 


<Af.||/||oo 


l^llLll^M/lli 


<Af.||/||oo 


MloWfWl 


<A|^||/||L 


\v\\lo 



E{{SMgiXi,Y^)f)<\\f\U\SMg\\l< 



\9\\l < ^IWfWlMl 



E{iSMh{X,,Y,)f) < ||/||oo||5M/i||i < \\f\\oo\\h\\l < A|- 
from previous calculations. Finally, 

Var(L(Xi,yi))<36A2.||/||^||r?||L. 



2 II ||2 

oo ir/iioo 



The last term of the Hoeffding's decomposition can also be controled : 
Lemma A. 4: Assuming the hypotheses of Theorem \3.2\ hold, we have 

Cov{UnK,PnL)=0. 

Proof: Since UnK et P„L are centered, we have 

Cov{UnK, PnL) = E{UnKPnL) 



E 

1 

n 




^ n n 

— - Y. K{Xj,Yj,Xk,Yk)YL{Xi,Yi, 

' j^k=l i=l 



'n?{n — 1) 



E{K{Xi,Yi,X2,Y2){L{Xi,Yi) + L(X2, Y2))) 



since K, L, Q and R are centered. 

The four previous lemmas give the expected result on the precision of On '■ 
Lemma A. 5: Assuming the hypotheses of Theorem \3.2\ hold, we have : 



D 



a 
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If m/n —7- 0, 



Otherwise, 



E(e„, -e? = o[^ 

n 



IE(^„ - ef < 72(mV) 



where 72 only depends on ||/||oo) Halloo (^nd Ay. 
Proof : Lemmas IA.21 IA.3I and IA.4I imply 

f ty I L J- J It 

Finally, for n large enough and a constant 7 G M, 



loo II 'I Woo' 



Var(^„)<7A2^||/||L||r?||L(-2+r 



m 1 

— + - 
n^ n 



Lemma lA.ll gives 

Bias^On) < A^||r?||^ ( sup |c 



2 

2ra \ ^ a2 ||„||2 / „,,„ l„ |2 ' 



■sifM 

and by assumption (supj^^j IqP) ~ m/n^ . If m/n — )• 0, then E(^„ — 9)'^ = 0(^)- 
Otherwise E(^„ — 0)^ < 72(m/n^) where 72 only depends on ||/||oo, Halloo and Ay. D 

The lemma we just proved gives the result of Theorem l3.2l when m/n does not converge 
to 0. Let us now study more precisely the semiparametric case, that is when E(^„ — 0)^ = 
O(^), to prove the asymptotic normality dH) and the bound in Q. We have 

^ [On -G)= V^iUnK) + V^iPnL) + ^/^i2A' B - A'CA). 

We will study the asymptotic behavior of each of these three terms. The first one is easily 
treated : 

Lemma A. 6: Assuming the hypotheses of Theorem \3.2\ hold, we have 

in probability when n — )■ 00 if m/n — t- 0. 

20 

Proof: Since \aii-\/nUnK) < rll^llLll/lloo^y("'' + 1)' V^UnK converges to in 

(n- 1) 

probability when n — )• 00 if m/n — t- 0. D 

The random variable PnL will be the most important term for the central limit theorem. 
Before studying its asymptotic normality, we need the following lemma concerning the 
asymptotic variance of y/n{PnL) : 



February 26, 2013 21:14 Journal of Nonparamctric Statistics EfficicntSA'ncwvcrsion 

REFERENCES 



29 



Lemma A. 7: Assuming the hypotheses of Theorem \3.S\ hold, we have 



nVariPnL)^A{f,r]) 



where 



Mf,v) = ^ 



9{x,yff{x,y)dxdy 



g{x,y)f{x,y)dxdy 



Proof: We proved in Lemma lA.31 that 



Var(L(Ai, Yi)) = 4Var[/i(Ai, Fi) + Sm9{XuYi] 
= 4VaT[Ai + A2 + A3] 
3 

We will show that Vi, j G {1, 2, 3}^, we have 



SMhiXi,Yi)] 



Cov{Ai,Aj) - Cij 



g{x,yff{x,y)dxdy- III g{x,y)f{x,y)dxdy 

<l[\\SMf-f\\2 + \\SMg-gh] (A2) 



where e^j = — 1 if i = 3 or j = 3 and ^ / j and eij = 1 otherwise, and where 7 depends 
only on ||/||oo, ||?/||oo and Ay. 

We shall give the details only for the case i = j = 3 since the calculations are siinilar for 
the other configurations. We have 



We first study the quantity 



Slf[h{x,y)]f{x,y)dxdy - / / g{x,yff{x,y)dxdy 



It is bounded by prout prout prout prout prout prout prout prout prout prout 



\Sh[hix,y)]f{x,y) - Slj[g{x,y)]f{x,y)\dxdy 



+ \Sh[9{x,y)]f{x,y) - g{x,yff{x,y)\dxdy 



< \\f\\oo\\SMh + SMghWSAih- SMg\\2 + \\f\\oo\\SMg + ghWSAig - g\ 



2- 
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Using the fact that Sm is a projection, this sum is bounded by 

ll/l|oo||/l + 5l|2||/i-5l|2 + 2||/||oo||5l|2||5'Af5-5l|2 

< ||/||oc(||/l||2 + |bl|2)||/l - 5II2 + 2||/||oc||<7||2||5m<7 - 5II2. 

1 /2 1 /2 

We saw previously that II5II2 < Ay||/||oo ||f?||oo and ll^lb < Ay||/||oo ||^||oo- The sum is 
then bounded by 

We now have to deal with \\h — g\\2'- 



{SmIXx, u) - f{x, u)) r]{x, u, y)du dxdy 



\h-g\\l 



< II { I {SMf{x,u) — f{x,u)) du\ I r]{x,u,y) du] dxdy 

</\lMU\SMf-f\\l 

Finally, the sum is bounded by 

2Ay||/||^2||^||^ (Ay||7?|U||SM/ " /lb + \\Sm9 " oh) 

Let us now study the second quantity 



SM[h{x,y)]f {x,y)dxdy 



g{x,y)f{x,y)dxdy 



It is equal to 



{SM[hix,y)] +g{x,y))f{x,y)dxdy 



{SM[h{x,y)] - g{x,y))f{x,y)dxdy 



By using the Cauchy-Schwarz inequality, it is bounded by 



2\\SMh + s'Ibll/lbll'S'M/i - gh 

+ ||5'I|2)(||5'm/i - Sugh + WSug - g\ 



< 

< 2Ay ||/||^2||^||^(||^ _ ^11^ ^ 11^^^^ _ ^11^) 

< 2Ay||/||^2||^||^ {AvMUlSMf - /II2 + WSug - gh) 

by using the previous calculations. Collecting the two inequalities gives ()A2p for i = j = 3. 
Finally, since by assumption \/t £ L'^{dfj,), \\SMt — t||2 — >• when n — )• 00, a direct 
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consequence of (|A2p is that 



Jim Yav{L{Xi,Yi)) 

gix,yffix,y)dxdy 



9{x,y)f{x,y)dxdy 



Mf,v)- 



We then conclude by noting that Var(^/n(i-'„L)) = Var(L(Xi, Yi)). D 

We can now study the convergence of ^/n{PnL), which is given in the following lemma: 
Lemma A. 8: Assuming the hypotheses of Theorem 1 3. S\ hold, we have 



V^P„L^MiO,Aif,v))- 



Proof: We first note that 



n Pn{2g) -2 g{x, y)f{x, y)dxdy ^ AA(0, A(/, r/)) 



where 5(a;,y) = / ri{x,y,u)f{x,u)du. 

It is then sufficient to show that the expectation of the square of 



R = y/Ti 



PnL - Pn{2g) -2 g{x,y)f{x,y)dxdy 



converges to 0. We have 



E(i?2) = Var(/?) 



nVar(P„L) + nVar(P„(2g)) - 2nCov(P„L,P„(25)) 



We know that riVaT(Pn{2g)) — )• A(/, rj) and Leninia fA.TI shows that nVar(i-'„L) 
Then, we just have to prove that 

lim nCov{PnL,Pn{2g)) = A(/,r?). 

We have 

nCov{PnL,Pn{2g)) = E{2L{X^,Y^)g{XuYi)) 
because L is centered. Since 



Hf,v)- 



L{Xi,Yi) = 2h{XuYi) + 2SMgiXuYi) - 2SMhiXi,Yi) - iA'B + 2A'CA, 
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we get 



nCov{PnL,Pni2g)) =4 // h{x,y)g{x,y)f{x,y)dxdy 



+4 / / SMgix,y)g{x,y)f{x,y)dxdy 

-4 / / SMh{x,y)g{x,y)f{x,y)dxdy - S^aibi // g{x,y)f{x,y)dxdy 



+4A'CA g{x,y)f{x,y)dxdy 



which converges to 4 ^yy ,(., yffix, y)dxdy - ^j j ,(x, .)/(x, ,)d.d.d, 
equal to A{f,r]). We finaUy deduce that 

V^P,L^AA(0,A(/,r/)) 

in distribution. 



which is 



D 



In order to prove the asymptotic normahty of 9n, the last step is to control the re- 
mainder term in the Hoeffding's decomposition: 

Lemma A. 9: Assuming the hypotheses of Theorem \3.2\ hold, we have 

^/^{2A'B - A'CA -9)^0. 
Proof: ./E{2A'B - A'CA - 6*) ^ is equal to 



2 // 9{x,y)SMf{x,y)dxdy 



SmUx, yi)SMf{x, y2)ri{x, yi,y2)dxdyidy2 



f{x, yi)f{x, y2)ri{x, yi,y2)dxdyidy2 



By replacing g we get 




SmUx, yi)f{x, y2)ri{x, yi,y2)dxdyidy2 



Sufix, yi)SMf{x, y2)vix, yi,y2)dxdyidy2 



fix, yi)f{x, y2)vix, yi , y2)dxdyidy2 
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With integral manipulation, we show it is also equal to 



SMf{x,yi){f{x,y2) - SMfix,y2))r]{x,yi,y2)dxdyidy2 



fix, y2){SMf{x, yi) - fix, yi))r]{x, yi,y2)dxdyidy2 

< V^AyM^ iWSMfhWSMf - fh + WfhWSMf - fh) 

<2V^Ay||/||2||||r/||oo||5A//-/||2 

1/2 



< 2VnAy||/||2||||??||oo ( sup |cip ) 



2Ay||/||2||||r/||ooA/-, 



which converges to when n — )• oo since m/n — )• 0. 

Collecting now the results of Lemmas IA.61 IA.7I and IA.9I we get ([8]) since 

^AA(0,A(/,77)) 



in distribution. We finally have to prove ([9]). Remark that 

/. \ 2 

nE iOn -^) = nBias2(6'„) + nVar(6'„) 

= nBias2(4) + nVar([/„K) + nVar(P„L) 
We previously proved that 

nBias^(§n) < AAy||7?||?o— for some A G M, 

n 



m 
n 



Lll^IlL— for some /xeM. 



D 



nYai{UnK) < /iA^| 
Moreover, ()A2p imply 

|nVar(P,L) - A(/,7?)| < 7 [||Sm/ - /II2 + WSmo - 5II2] , 

where 7 is a increasing function of H/HooJI^Hoo and Ay. We then deduce ^ which ends 
the proof of Theorem 



A. 3. Proof of Theorem [3IE 

To prove the ineq uality we will us e the w ork of llbragimov and Khasihinskiil ( 199ll ) (see 
also chapter 25 of IVan Per VaartI ( 19981 )) on efficient estimation. The first step is the 
computation of the Frechet derivative of 9{f) at a point /q. Straightforward calculations 
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show that 



o{f)-eifo) 



2 / 7p{x,y,z)fo{x,z)dz 



ifix,y) - fo{x,y))dxdy 



+ 0{ {f{x,y)-Ux,y)Ydxdy 



from which we deduce that the Frechet derivative of 9[f) at /o is 

Q'{h) """^ \2 / V'(2;,2/,z)/o(a;,z)(i2,n\ (n G L^{dxdy)), 



where (•,• ) is the scalar product i n L'^ (dxdy). We can now use the re- 
sults of llbragimov and Khasrhinskiil (Il99ll ). Denote H{fo) = H{fo) = 

<u ^ L'^{dxdy), ff u{x,y)y/ fQ{x,y)dxdy = > the set of functions in L'^{dxdy) or- 
thogonal to ^/f^, Projji^(j^) the projection on H{fo), An{t) = {^/fo)t/y/n and PJ^' 
the joint distribution of (Xi,...,X„) under /q. Since here Xi,...,X„ are i.i.d., 
< Pr" , / € <S > is locally asymptotically normal at all points fo € £ in the direction 

H{fo) with normalizing factor A„(/o). Ibragimov and Khas'minskii result say that under 
these conditions, denoting Kn = -Bn^'(/o)^nPi'oj//(/p) with Bn{u) = \/nu, if i^„ — )• K 

weakly and if K{u) = {t,u), then for every estimator 9n of 0(f) and every family V(/o) 
of vicinities of /o, we have 



{V(/o)} n^oo /gv(/„) ^ y^ 



Here, 



Kn{u) = V^e'ifo) ■ -^V^ProJH(^„)H = 9'{fo) ■ (^0 U - ^/To J u^/f^ 



does not depend on n and 



K(u) 



2 / %l){x,y,z)fo{x,z)dz 



Vfo{x,y) 



u{x, y) - \/foix,y) / uV7o ) dxdy 



II 



2 / tlj{x,y,z)foix,z)dz 



Vfoix,y)u{x, y)dxdy 



2 / ^{x,y,z)fo{x,z)dz 



fo{x,y)dxdy / ua//, 



(t,n) 
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where 



t{x,y) 



2 / '4){x,y,z)fQ{x,z)dz 



VMx,y) 



2 / 7p{x,y,z)fo{x,z)dz 



fo{x,y)dxdy \/foix,y) 



The semiparametric Cramer- Rao bound for our problem is ||t"^ 



L^ (dxdy) 



\L^{dxdy) 



l2 



i^{x,y,z)fo{x,z)dz 



fo{x,y)dxdy 



'^ix,y,z)fo{x,z)dz 



fo{x,y)dxdy 



4 // go{x,yffo{x,y)dxdy-A{ // go{x,y)fo{x,y) 



where go{x,y) = / 'ilj{x,y,z)fQ{x,z)dz. Finahy, we recognize the expression of A(/o,V') 
given in Theorem! 



A. 4. Proof of Theorem 3-4 



We will first control the remainder term r„ : 



1 



rn = -F'"(e)(i-e)'. 





Let us recall that 



F"'{0 




/(x, y)dy 



Cf{x,y) + {l-Of{x,y)dy'^ 
[{7h{x) - Lp{y)) [m{x) - ip{z)) (m(x) - ip{t)) 

f{x, y)dy ) 'ii) (f(^, x)) - 2>[rh{x) - (p{y)) (m(x) 



V^iz)) 



[Cfix,y) + il-Ofix,y)]dy]i>iriC,x)) 



f{x, y) - fix, y) ] ( f{x, z) - /(x, z) 



f{x, t) — f{x, t) j dxdydzdt 
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Assumptions A2 and A3 ensure that the first part of the integrand is bounded by a 
constant /i : 



r„,<^^ //// \f{x,y)-f{x,y)\\f{x,z)-f{x,z)\ 



\f{x,t) — f{x,t)\dxdydzdt 



\fix,y)- fix,y)\dy] dx 



1 
<i/iA2, // \f{x,y)-f{x,y)\^dxdy 



by the Holder inequahty. Then E{Tl) = 0(E[(/ \f - /|3)2]) = 0(E[||/ - /||6]). Since / 
verifies assumption A2, this quantity has order 0{n^ ). If we further assume that ni ~ 
ra/log(n) and A > 1/6, we get -E(r^) = o(-), which proves that the remainder term r„ 

is neghgible. We will now show that ^/E ('f„ - T{f)\ and ^„ = ;^ Ej=i H{f, Xj,Yj) - 

JJ H{f,x,y)f{x,y)dxdy have the same asymptotic behavior. The idea is that we can 
easily get a central limit theorem for Z„ with asymptotic variance 



C{f) = J J H{f, X, yff{x, y)dxdy - (JJ H{f, x, y)f{x, y)dxdy 



which imply both (llOp and (jlip (we will show at the end of the proof that C{f) can be 
expressed such as in the theorem). In order to show that ^/n ( T„ — T(/) J and Z^ have 
the same asymptotic behavior, we will prove that 



R = ^/n 



Tn-T{f)-{^J^H{f\X,,Y,)- JJ 



Hif,x,y)f{x,y)dxdy 



has a second-order moment converging to 0. Let us note that R = Ri + R2 where 



Ri 



R2 



V^ [fn - T{f) 

Y, H{f, X,,Yj) - II H{f, x, y)f{x, y)dxdy 
— X; (hU. Xj.Yj) - [[ Hif, X, y)fix, y)dxdy) 



n2 ^, 



i=i 



l-f2(H{f,Xj,Yj)- II H{f,x,y)f{x,y)dxdy 
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We propose to show that both K{Rf) and ]E(i?2) converge to 0. We can write i?i as 
follows : 



Ri = -\fn 



where 



K{f,x,y,z 



Q'-Q' + r„ 



Q' = /// K{f,x,y,z)f{x,y)f{x,z), 
1 tlj{m{x)) 



J fix, y)dy 



rh{x) — (p{y)) (m(x) — ip{z)) 



and Q' is the corresponding estimator. Since E (r^) = o(l/n), we just have to control 



the expectation of the square of ^/n 



Q'-Q' 



Lemma A. 10: Assuming the hypotheses of Theorem \3.4\ hold, we have 

lim nE (q' - Q') = 0. 

n^oc \ / 

Proof: The bound given in ([9]) states that if |M„|/n — )• we have 
r / . \ 2 .1 

g{x,y)f{x,y)dxdy 



Q'-Q') \f 

9ix,yff{x,y)dxdy 

<7l(ll/l|oo,||V'l|oo,Ay)^'^" 



n 



+ \\SMf-fh + \\SM9-gh 



where g{x,y) = / K{f,x,y,z)f{x,z)dz. By deconditioning, we get 



nE 



{q' - Q' 



g{x,yff{x,y)dxdy- // g{x,y)f{x,y)dxdy 



<7l(ll/l|oo,||V'l|oo,Ay) 



|M„ 



n 



\\SMf-fh + E{\\SMg-9\ 



Note that 



E (||5m5 - gh) < E iWSMg - SMgh) + E (||5m5 - gh) 

<E{\\g-g\\2)+E{\\SMg-g\\2) 
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where g{x,y) = / K{f,x,y,z)f{x,z)dz. The second term converges to since g G 
L?{dxdy) and Vt G L?{dxdy), J{SMt - t)'^dfi -> 0. Moreover 



19- gh 



< 



{x, y) - g{x, y)f f{x, y)dxdy 



Kif, X, y, z) - K{f, X, y, z) ] f{x, z)dz 



f{x,y)dxdy 



Kif,x,y,z)- K{f,x,y,z)) dz 



f{x,zfdz 



<A^||/||L 



<5A? 11^113 



YWJ lloo 



f{x,y)dxdy 



K{f,x,y,z)-K{f,x,y,z)) dxdz 



{f{x,y) - f{x,y)fdxdy 



for some constant 5 by applying the mean value theorem to K{f, x, y, z) — K{f, x, y, z). 
Of course, the bound S is obtained here by considering assumptions Al, A2 and A3. 
Since E(||/ — /II2) — )■ 0, we get E (||^ — (7II2) — )■ 0. Let us now show that the expectation 
of 



g{x,yff{x,y)dxdy 



g{x,y)f{x,y)dxdy 



converges to 0. We will only develop the proof for the first term : 



// g{x,yff{x,y)dxdy- // g{x,yff{x,y)dxdy 
< // loix, yf - g{x,yf\ f{x,y)dxdy 



< ^ / / {9{x,y)- g{x, y)f dxdy 
<M\9-9\\l 



for some constant A. By taking the expectation of both sides, we see it is enough to show 
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that E (11^ — (7II2) — >• 0, which is done exactly as above. Besides, we can verify that 



9{x,y) = / K{f,x,y,z)f{x,z)dz 

2{Jf{x,y)dy)^ ^ ' ^"-"'^ 
m{x) / f{x,z)dz— / ip{z)f{x,z)dz 

0, 



which prove, that the expectation of// s(.. yff(., y)i.iy converge, to 0. Similar con- 

siderations show that the expectation of the second term I / / g{x, y)f{x, y)dxdy J also 
converges to 0. We finally have 



lim nE g' - Q' = 0. 



Lemma ETTO] imply that E(-Rf) -> 0. We will now prove that E(i?|) -> : 



D 



n 



mR^) = _E 

^2 



Hif,x,y) - H{f,x,y)) f{x,y)dxdy 



n2 



1 2 



H{f,x,y)f{x,y)dxdy - // H{f,x,y)f{x,y)dxdy 



The same arguments as before (mean value theorem and assumptions A2 and A3) show 
that E(i?2) ~^ 0- At last, we can give another expression for the asymptotic variance : 



Cif) 



llHif,x,yffix,y)dxdy-nJH{f,x, 



y)f{x,y)dxdy 



We will prove that 



C(/) = E Var((^(y)|X) V' (E(l'l^)) + Var (V' (E((^(y)|X))) 
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Remark that 



H{f,x,y)f{x,y)dxdy = jj [\ip{y) - m{x)]ij{m{x)) + ^{m{x))j f{x,y)dxdy 

m{x)ip{'m{x))f{x,y)dxdy— // m{x)'ip{'m{x))f{x,y)dxdy 



+ 'ip{m{x))f{x,y)dxdy 

E(V^(EMy)|X))). (A3) 



Moreover, 



H{f,x,y) = [ip{y) — m{x)] ip{m{x)) + %l){ra{x)) +2[(p{y) — m{x)]TJ){m{x))%lj{m{x)) 
= (p{y) ip{m{x)) + m{x) ip{m{x)) — 2ip{y)m{x)'\p{m{x)) 
+il){m{x)) + 2 [i^iy) — m{x)] ■ip{m{x))il){m{x)). 



We can then rewrite / / H{f,x,y) f{x,y)dxdy as: 



(f{y) ^(m(x)) f{x,y)dxdy+ // ?7i(x) ^{m{x)) f{x,y)dxdy 
-2 // ip{y)m{x)ip{m{x)) f{x,y)dxdy+ // ^(?7i(x)) f{x,y)dxdy 
-2 / / ip{y)ip{m{x))il){m{x))f{x,y)dxdy — 2 // m{x)ip{m{x))ilj{m{x))f{x,y)dxdy 

v{x)ip{rn{x)) f{x,y)dxdy— // m{x) 'il){m{x))f{x,y)dxdy+ jj 'il){rn{x)) f{x,y)dxdy 

[v{x) — m{x) ~\ip{m{x)) +'ilj{m{x)) j f{x,y)dxdy 
E ( [v{X) - m{Xf] TPimiXyf) + E {4){m{X)f) 
E([Eicp{Yf\X)-KiipiY)\Xf] UiEiipiY)\X))Y) + E (V^(EMy)|X))2) 



E Var((p(y)|X) ijj (K{Y\X)) + E {ij(E{ipiY)\X)f) 



where we have set v{x) = J ip{y)'^f{x,y)dy/ J f{x,y)dy. This result and ()A3P give the 
desired form for C{f) which ends the proof of Theorem 13. 
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A. 5. Proof of Theorem lEIE 

We follow the proof of Theorem 13.31 Assumptions A2 and A3 imply that 

T{f) - r(/o) = II ( [<f{y) - moix)] ij{mo{x)) + ^l^{moix)) 



[f{x,y) - foix,y)yxdy + O ( if - fo) 



where mo{x) = J ip{y)fo{x, y)dy/ J fo{x, y)dy. This result shows that the Frechet deriva- 
tive of r(/) at /o is r'(/o) • h = {H{fo, •), h) where 

H{fo,x,y) = \^[<piy) - mo{x)]ij{mo{x)) + ip{mo{x))j . 

We then deduce that 

K{h)=T'{fo)-(^/Tolh-^olh^,)) 

= I H{fo, ■)^oh - I H{fo, ■)VTolh^o 

= {t,h) 
with 

t = H{fo,-)^/h- nHifo,-)fo] VTo- 
The semiparametric Cramer-Rao bound for this problem is thus 



\ L^ (dxdy) 



I H{hrfh-{^l H{ 



(/o,-)/o =C(/o) 



where we recognize the expression of C{fQ) in Theorem [3? 



